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Any references cited hereafter are incorporated by reference to the maximum 
extent allowable by law. To the extent a reference may not be fully incorporated herein, 
it is incorporated by reference for background purposes and indicative of the knowledge 
of one of ordinary skill in the art. 

5 

BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 
10 This application relates to the field of managing TCP communications. 

DESCRIPTION OF RELATED ART 

The field of managing TCP communications has a history of rapid development. 

15 The speed and volume of TCP communications have increased dramatically every few 
years. The network infi-astructure responsible for carrying TCP communications have not 
typically been required to process persistent state information. The processing of 
persistent state information such as TCP connection information has typically been 
handled by general purpose processors or speciahzed processors for routing (i.e., stand- 

20 alone network processors). However, the volume of TCP communications has increased 
so dramatically that it is not uncommon for services at the edge of a data center to require 
the management and processing of one million or more TCP connections. Firewalls, load 
balancers, etc., could also be enhanced if there were a capability to either terminate or 
shadow TCP connections at wire speeds. 

25 Figure 1 shows a TCP state management system of the prior art including a shared 

memory subsystem 12 and one or more general purpose processors 14. Each general 
purpose processor 14 includes memory 16 and processor 18. One advantage of using 
general purpose processors 14 is the achievement of a great deal of flexibility in the TCP 
protocol state management system. However, general purpose processors 14 are not as 
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efficient as would be desired considering the great number of TCP communications 
typically needing to be processed today. The problem is expected to intensify in the 
future. Another disadvantage is that a router may be required to ensure transmission of 
each packet to the proper general purpose processors 14 for processing. Furthermore, a 
5 layer of software is typically added to handle the coherency conditions created by 

communication and data sharing between the general purpose processors 14. Complex 
coherency conditions can thereby be produced. 

BRIEF SUMMARY OF THE INVENTION 

10 

An embodiment of the present invention is a state decision subsystem (SDS) 
including an inload module, a simple programmable entity (SPE), at least one SPE 
memory, an unload module, and a coherency module. The inload module reads state 
information fi-om a memory subsystem — the state information corresponding to TCP 

1 5 packets that are to be processed. In addition, the inload module writes contexts to the 
SPE memory. Each context includes both a TCP packet and its corresponding state 
information. The SPE reads each context from the SPE memory, processes the context, 
and writes the processed context to the SPE memory. The processed context includes 
both a processed TCP packet and its corresponding processed state information. 

20 Furthermore, the unload memory reads the processed context and writes the processed 
state information to the memory subsystem. Finally, the coherency module compares 
contents of the inload module and the unload module, and if the contents of each 
correspond to the same TCP connection, the coherency module replaces the state related 
content of the inload module with the state related content of the unload module. 

25 A variation of the embodiment includes two SPE memories that cooperate to 

operate as a ping-pong buffer. In another variation, the inload module includes an inload 
queue for queuing contexts. Similarly, the imload module may include an unload queue 
for queuing processed context. If such queues are implemented, then the coherency 
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module will need to compare the entire contents of the queues in order to maintain 
coherency. 

One advantage of the present invention in achieving increased efficiency is 
realized to a greater extent by implementations that prefetch states for a large number of 
5 packets. Significant efficiency improvements, due in large part to parallelizing and 
offloading of data movements, are realized directly in proportion to the number of 
packets for which state infomiation is prefetched. The efficiency gains can significantly 
outweigh the additional burden of maintaining coherency among a larger number of 
prefetched states. In another variation of the embodiment, the inload module classifies 

1 0 TCP packets by packet type and includes packet type information as part of each context. 
The packet type information can be encoded as a bit vector. 

In yet another variation of the embodiment, each TCP packet includes a packet 
sequence number and a length value. The packet sequence number corresponds to the 
sequence number of the first byte of the packet. The length value corresponds to the 

1 5 number of bytes contained in the packet. Consequently, the state information 

corresponding to each TCP packet indirectly includes a value for a next packet sequence 
number that designates the sequence number of the next TCP packet to be processed. In 
processing a TCP packet, the inload module determines whether the packet sequence 
number of the TCP packet corresponds to the next packet sequence number of the TCP 

20 connection corresponding to the TCP packet. If not, the inload module stores the TCP 
packet for processing after processing earlier sequenced packets corresponding to the 
same TCP connection. 

In a fiirther embodiment of the present invention, the SPE recognizes when two 
contexts are loaded that correspond to the same connection. In that case, the state 

25 information of the second context loaded is ignored because the first context's state 
information, including any modifications, is the more current. 

Physical arrangement of embodiments of the present invention can very widely. 
For example, in some embodiments, SPE memory is implemented in the same physical 
device as the corresponding SPE, while in other embodiments the reverse is true. 
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Many other variations of the invention are possible. For example, another 
embodiment of the present invention is an intranet having at least one SDS as described 
above that processes packets corresponding to a number of TCP connections such that 
each connection has its terminations inside of the intranet. Other similar embodiments 
5 handle TCP connections each having one termination outside of the intranet. Still other 
similar embodiments handle some TCP connections having both terminations within the 
intranet and some TCP connections having one termination outside of the intranet. 
Another embodiment of the present invention is a load balancer that includes at least one 
SDS as described above. Furthermore, another embodiment of the present invention is a 
10 firewall having at least one SDS as described above. Further embodiments of the present 
invention could include any other devices that process TCP connections with at least one 
SDS, including for example, intrusion detection systems, routers, web-caching devices, 
LANS, and SANS. 

Yet another embodiment of the present invention is a TCP acceleration system 
15 including an ingress state decision subsystem (SDS), an egress SDS, and a timer SDS that 
performs timing related tasks. The ingress SDS reads state information from a memory 
subsystem. The state information corresponds to incoming TCP packets. The ingress 
SDS also processes the incoming packets according to the state information and writes 
updated state information to the memory subsystem. The egress SDS reads state 
20 information from the memory subsystem. The state information corresponds to outgoing 
TCP packets. The egress SDS processes the outgoing packets according to that state 
information and writes updated state information to the memory subsystem. The timer 
SDS reads state information from the memory subsystem to perform time related 
processing of the state information and corresponding TCP connections. 
25 Another embodiment of the present invention is an intranet having at least one 

TCP acceleration system as described above, that processes packets corresponding to a 
number of TCP connections each having a termination outside of the intranet. Another 
embodiment of the present invention is a load balancer that includes at least one TCP 
acceleration system as described above, and yet another embodiment of the present 
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invention is a firewall including at least one TCP acceleration system as described above. 
Furthermore, embodiments of the present invention could include any other device that 
processes TCP connections with a TCP acceleration system, including for example, 
intrusion detection systems, routers, web-caching devices, LANS, and SANS. Therefore, 
5 as will be apparent to those skilled in the art, acceleration of TCP processing according to 
the present invention within the context of the hitemet or other networking environment 
will fall within the scope of the present application. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 

The following drawings form part of the present specification and are included to 
further demonstrate certain aspects of the present invention. The figures are not 
necessarily drawn to scale. The invention may be better understood by reference to one 
or more of these drawings in combination with the detailed description of specific 
1 5 embodiments presented herein. 

FIG. 1 shows a TCP state management system of the prior art, in accordance with 
an embodiment of the present invention. 

FIG. 2 shows a TCP state management system, in accordance with an 
embodiment of the present invention. 
20 FIG. 3 shows a TCP state management system having a ping-pong buffer, in 

accordance with an embodiment of the present invention. 

FIG. 4 shows a TCP state management system having a ping-pong buffer and an 
inload module with packet classification personality, in accordance with an embodiment 
of the present invention. 
25 FIG. 5 shows a TCP state management system having multiple state decision 

subsystems, in accordance with an embodiment of the present invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 



As noted above, the present invention builds on some concepts analogous to those 
disclosed by two commonly owned and previously filed U.S. patent applications. The 
5 application identified by application number 10/068,295 and entitled "Application- 
Specific Information-Processing Method, System, and Apparatus" discloses an 
information-processing method for application-specific processing of messages. In that 
method, a message is received. Then, whether the message is in a selected application 
format is ascertained. If the message is not in the selected application format, it is routed 
10 to a next location. However, if the message is in the selected application format, it is 

routed to a selected application processor, processed by the processor, and then routed to 
the next location. 

The application identified by application number 10/078,253 and entitled 
"Controller Architecture and Strategy For Small Discontiguous Accesses to High-Density 

15 Memory Devices'' and its divisional applications identified by application numbers 

10/640,499 and 10/640,462 and respectively entitled "Network and Networking System 
for Small Discontiguous Accesses to High-Density Memory Devices" and "Memory 
Request Handling Method for Small Discontiguous Accesses to High-Density Memory 
Devices" disclose a memory architecture that use memory devices that would normally 

20 be considered disadvantageous, but by accommodating the data input, output, and other 
peripheral controller services, overall performance in this mode is optimized. The 
surprising result is that even though the choice of memory is inappropriate for the task 
based on the precepts of the prior art, the overall memory system is effective. One 
example of a normally disadvantageous approach that is effective in connection with one 

25 or more embodiments of the present invention is bank switching in DDR-SDRAM, 
thereby achieving feasibility without resort to, for example, SRAM, 

Among other advantages achieved, both of the referenced applications improve 
performance by resolving the presented problem with a solution more closely tailored to 
the presented problem than the prevailing prior art approach. 
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FIG. 2 shows a TCP state management system having a shared memory 
subsystem 29 and a state decision subsystem 20. Shared memory subsystem 29 stores 
state information for TCP connections that are to be processed by state decision 
subsystem 20. The state decision subsystem 20 includes an inload module 24 and an 
5 unload module 26, each in communication with the shared memory subsystem 29, such 
that inload module 24 reads state information from the shared memory subsystem 29 and 
unload module 26 writes state information to the shared memory subsystem 29. 
Coherency checking is performed by simple coherency checking module 28. The module 
28 reads information from inload module 24 and unload module 26 to compare state 

10 information. If it is determined by simple coherency checking module 28 that state 

information corresponding to the same packet is present in both module 24 and 26 then 
the state information corresponding to that packet contained in unload module 26 is 
written over the state information corresponding to that packet contained in inload 
module 24. The state information and corresponding packet information is written by 

1 5 inload module 24 to SPE memory 22. SPE 21 reads such state and packet information 
from memory 22 and processes it accordingly, writing the result to memory 22. Unload 
module 26 then reads the result from memory 22. It should be apparent that the simple 
programmable element 21 is not required to address coherency issues. Rather, it simply 
processes packets according to accompanying state information. In the preferred 

20 embodiment, the inload module and the unload module, as well as the simple coherency 
checking module are implemented as hardware that improves performance significantly. 
Those fiinctions in the prior art would typically be performed by software resulting in 
slower performance. 

Turning to FIG. 3, a TCP state management system having a ping-pong buffer is 

25 shown. The system includes state decision subsystem 30 and shared memory subsystem 
39. Shared memory subsystem 39 stores state information for TCP connections that are 
to be processed by state decision subsystem 30. The inload module 34 of subsystem 30 
reads state information from subsystem 39. Central coherency checking module 38 then 
compares the content of inload module 34 with the contents of unload module 36. If state 
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information is found to be contained in inload module 34 as well as unload module 36 
that corresponds to the same TCP packet, then the state information from unload module 
36 is written over that state information of inload module 34. As in the system of FIG. 2, 
in the system of FIG. 3, inload module 34, simple coherency checking module 38 and 
5 unload module 36 are implemented in hardware in a preferred embodiment. After the 
simple coherency checking module 38 has finished, the inload module 34 passes state 
information and corresponding packet information to one of SPE memory 32 or SPE 
memory 33. The two contexts A and B produce a ping-pong buffer 32, 33 configured to 
improve the efficiency of state decision subsystem 30 by allowing SPE memory 32 to be 

1 0 read from while SPE memory 33 is written to or vice-versa. So, at any given time, 
context will be A or B for this system. 

For example, if the present state of the system is context A, then inload module 34 
writes the state and packet information to SPE memory 32. Subsequently, SPE 31 reads 
the state and packet information from SPE memory 32, processes the packet information 

1 5 and state information accordingly and then writes the processed state and packet 

information to SPE memory 32. Concurrently, inload module 34 writes new state and 
packet information to SPE memory 33. Thus when SPE 31 finishes writing processed 
state and packet information to SPE memory 32, it is quickly able to read new state and 
packet information from SPE memory 33 without waiting a significant period of time. 

20 Continuing with the operation of the system, unload module 36 reads processed state and 
packet information from SPE memory 32 and then writes the processed state information 
to subsystem 39. 

The use of a ping-pong buffer is a straight-forward optimization that avoids the 
SPE 31, having to wait for context to be loaded to or unloaded from its memory. 
25 Alternatively, more than two SPE memories could be used to fiirther improve the 
performance of the system. 

Turning now to FIG. 4, a TCP state management system having a ping-pong 
buffer and an inload module with packet classification fimctionality is shown. The 
system of FIG. 4 includes state decision subsystem 40 and shared memory subsystem 51. 
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Shared memory subsystem 51 stores state information for TCP connections that are to be 
processed by state decision subsystem 40. State decision subsystem 40 includes many 
components corresponding to state decision subsystem 30. Such corresponding 
components, while not necessarily identical, perform similar functions in both 
5 subsystems. The corresponding components include simple programmable element 
(SPE) 41 corresponding to SPE 31. SPE memory 42 corresponds to SPE memory 32. 
SPE memory 43 corresponds to SPE memory 33. Ping-pong buffer 42, 43 corresponds to 
ping-pong buffer 32, 33. Unload module 45 corresponds to unload module 36. Simple 
coherency checking module 46 corresponds to simple coherency checking module 38. 

10 A difference between the system of FIG. 4 and the system of FIG. 3 is to be found 

in the additional functionality of inload module 44 compared to inload module 34. Inload 
module 44 reads state information from shared memory subsystem 51. The state 
information is queued in packet state queue 50 while packet information is queued in 
packet input queue 49. Packet information includes information designating the type of 

15 packet. For example, a packet may be a type ACK, SYN, FIN, etc. A packet type 
classifier 48 preferably implemented in hardware determines the packet type from the 
packet information and attaches the packet type to the state information, thus inload 
module assembler 47 assembles the context from state information, type information, and 
packet information. One alternative would be to encode the type information in a bit 

20 vector. Then the inload module 44 would write the context to the appropriate SPE 
memory 42 or 43. 

FIG. 5 shows a TCP state management system having multiple state decision 
subsystems (SDS). Shared memory subsystem 59 stores state information for TCP 
connections that are to be processed by state decision subsystems 52, 53, and 54. Ingress 
25 SDS 52 reads from and writes to shared memory subsystem 59. Egress SDS 53 reads 

from and writes to shared memory subsystem 59 and timer SDS 54 reads from and writes 
to shared memory subsystem 59. In a preferred embodiment, the ingress state decision 
subsystem 52 handles all of the incoming packets from the network and makes decisions 
related to opening and closing connections. The egress state decision subsystem 53 
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handles all of the outgoing packets to the network. The timer state decision subsystem 54 
handles state updates on the connections related to timed events such as 2MSL, idle 
connections, etc. In other embodiments, the overall process is divided into other logical 
portions than ingress, egress and timer. Furthermore in yet other embodiments, the 
5 division of functions among the multiple state decision subsystems may be made 
arbitrarily. 

An altemate implementation of a TCP state management system is shown in FIG. 
6. Architecture is partitioned so as to customize the handling of specific sub-problems. 
Efficiency in handling the specific sub-problems can thereby be improved. Further, the 

10 specific sub-problems are thereby handled in parallel in some embodiments, resulting in 
further overall performance improvement. 

The specific system shown in FIG. 6 includes state decision subsystems 60 and 67 
and shared memory subsystem 69. Shared memory subsystem 69 stores state information 
for TCP connections that are to be processed by state decision subsystems 60 and 67. 

1 5 State decision subsystem 60 includes an inload module 65, an unload module 66, a shared 
memory subsystem 63, and state decision subsystems 61 and 62. Moreover, other 
alternative embodiments falling within the scope of the present application include 
multiple state decision subsystems similar or identical to 60 or 67. 

According to the specific sub-problems to be addressed, some embodiments 

20 implement nesting of one or more state decision subsystems having the same or similar 
architecture within a state decision subsystem, while others do not. 

As apparent to one of ordinary skill in the art, the architecture described provides 
improved flexibility to implement a complex protocol such as TCP and dedicated 
hardware to accelerate the system to higher performance levels. Efficiency of the system 

25 is improved in part because the simple programmable elements are not required to 

intelligently move data, needing only to have the context of a single packet to make all of 
the decisions for that packet. For example, a simple programmable element provided 
with state information and packet information is able to process that packet without the 
overhead of coherency issues. 
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Any element in a claim that does not explicitly state "means for" performing a 
specified function, or "step for" performing a specific function, is not to be interpreted as 
a "means" or "step" clause as specified in 35 U.S.C. § 1 12, If 6. In particular, the use of 
"step of in the claims herein is not intended to invoke the provision of 35 U.S.C. § 1 12, 
5 116. 

It should be apparent from the foregoing that an invention having significant 
advantages has been provided. While the invention is shown in only a few of its forms, it 
is not limited to only those forms but is susceptible to various changes and modifications 
without departing firom the spirit or scope thereof. 

10 For example, the inload module 44 of FIG. 4 is shown being implemented in 

connection with ping-pong buffer 42, 43. But an inload module similar to inload module 
44 could easily be implemented in an embodiment of the present invention without a 
ping-pong buffer being implemented. 

Additionally, shared memory subsystems of the embodiments have been shown as 

15 single memories apart from the corresponding state decision subsystems. But any single 
shared memory subsystem could be implemented across multiple memory devices. Also, 
a shared memory subsystem can be implemented in the same physical device in which 
the corresponding state decision subsystem is implemented. 

Furthermore, any of the above-described state decision subsystems can be 

20 implemented as a single physical device, or its functions and modules can be 

implemented across multiple physical devices having appropriate interconnectivity. 
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